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Abstract 

In the 60's Shapley provided an example of a two player fictitious game with periodic 
behaviour. In this game, player A aims to copy B's behaviour and player B aims to play 
one ahead of player A. In this paper we generalize Shapley's example by introducing an 
external parameter. We show that the periodic behaviour in Shapley's example at some 
critical parameter value disintegrates into unpredictable (chaotic) behaviour, with players 
dithering a huge number of times between different strategies. At a further critical param- 
eter the dynamics becomes periodic again, but now both players aim to play one ahead 
of the other. In this paper we adopt a geometric (dynamical systems) approach and show 
that GENERICALLY the dynamics of fictitious play is essentially continuous and uniquely 
defined everywhere except at the interior equilibrium. Here we concentrate on the periodic 
behaviour, while in the sequel to this paper we shall describe the chaotic behaviour. 

1 Introduction 

Suppose that a two-player finite game in normal form is repeated infinitely often. Suppose further 
that, at each stage of the repeated game, each player plays a best response to the empirical 
distribution of the past play of her opponent. Then the state of the repeated game at any given 
stage can be summarized by the pair consisting of the empirical distribution of past actions 
of player A and the empirical distribution of past actions of player B, and this state evolves 
according to a dynamical system. This dynamical system is known as fictitious play. 

Fictitious play arises in many different guises in economics. It was originally proposed as an 
algorithm for finding equilibrium, see Brown (1949), Brown (1951a) and Brown (1951b). It was 
later reinterpreted as a model of boundedly rational learning in interactive settings, see Chapter 

2 of Fudenberg & Levine (1998) and the references cited there. Several modifications have been 
introduced, including: (i) a modification in which players do not play exact best responses but 
instead choose actions with a probability that increases with their payoffs, see Cowan (1992) and 
Fudenberg & Levine (1995); (ii) a modification in which each player is replaced by a population 
of players whose payoffs differ by a small random perturbations, see Ellison & Fudenberg (2000); 
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and (iii) a modification in which players play exact best responses with some probability and 
experiment at random with some probability, see Fudenberg & Levine (1998) and the references 
cited there. Finally, there is a discrete-time version, in which the underlying normal-form game 
is played for each t <G {0, 1, 2, ...}, and a continuous-time version, in which the underlying normal- 
form game is played for each t G [0, +oo). 

Much effort has been devoted to finding sufficient conditions under which fictitious play 
converges to an equilibrium from all starting points. For example, fictitious play has been shown 
to converge in: (i) zero-sum games, see Robinson (1951), Hofbaucr (1995) and Harris (1998); 
(ii) generic 2 x n nonzero-sum games (see Miyasawa (1961) and Metrick & Polak (1994) for the 
2x2 case; Sela (2000) for the 2x3 case; and Bcrger (2005) for the general case); (iii) potential 
games (see Monderer & Shapley (1996) for the case of weighted potential games and Berger 
(2007) for the case of ordinal potential games); (iv) dominance-solvable games (see Milgrom & 
Roberts (1991)) and (v) various classes of supermodular games (see: Krishna (1992) for the case 
of supermodular games with diminishing returns; Hahn (1999) for the case of 3 x 3 supermodular 
games; and Bcrger (2007) for the case of quasi-supcrmodular games with diminishing returns, 
the case of quasi-supermodular 3 x n games and the case of quasi-supermodular 4x4 games) . 

Some effort has also been devoted to understanding fictitious play in games in which fictitious 
play does not necessarily converge to an equilibrium. One approach is by way of examples. The 
first contribution to this strand of the literature was made by Shapley (1964), who identified a 
generic 3x3 nonzero-sum game for which fictitious play has a stable limit cycle. Three more 
recent contributions are: Jordan (1993), who constructed a 2 x 2 x 2 game for which fictitious 
play has a stable limit cycle; Gauncrsdorfer & Hofbauer (1995), who established an interesting 
relationship between the replicator dynamics and fictitious play in three examples in which 
fictitious play converges to a limit cycle (namely Shapley's example, Jordan's example and the 
rock-scissors-paper game); and Cowan (1992), who identified a difficult numerical example in 
which fictitious play exhibits chaotic behaviour (in the sense that the flow contains a subshift of 
finite type). Our work is closest in spirit to Berger (1995). 1 

Another approach is taken by Krishna & Sjostrom (1998), building on earlier work of Roscnmiiller 
(1971). In this interesting paper, they show that generically (i.e. for Lebesgue almost all pay-off 
matrices) , fictitious play cannot converge cyclically to a mixed strategy equilibrium if both players 
use more than two pure strategies. In other words, if fictitious play converges to a mixed strategy 
equilibrium with both players using more than two strategies then the choice of strategies cannot 
follow a cyclic pattern. 2 

The purpose of this and a sequel to this paper is a detailed analysis of the following family 
of 3 x 3 games with utilities v A and v B determined by the matrices 



1 We would like to thank the referee for bringing this unpublished work to our attention. Berger considers a 
family of symmetric bimatrix games depending on a parameter k such that (i) for k 6 (—2, 0) has a Shapley orbit 
which degenerates as k — > 0, and (ii) for k > 0, as in our case, there exists a hexagonal orbit along which both 
players are indifferent between (at least) 2 strategies. Berger observes similar 'chaotic' numerical phenomena as 
in Subsection 3.5 of our paper, and also shows the existence of an additional periodic orbit of saddle-type. 

2 For their result Krishna and Sjostrom only (need to) consider the situation where at any given moment only 
one of the players is in indifferent. As becomes clear from our paper (and its sequel), it is precisely near the set 
where both players are simultaneously indifferent that much of the interesting behaviour happens (and this set 
'organises' the local dynamics). 
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which depend on a parameter [3 G (— 1, 1) (and best response dynamics given by the differential 
inclusion (2.2)). We will show that for some parameters (3 play becomes cyclic and for others it 
becomes erratic. 

For (3 = 0, this game is equal to Shapley's: irrespective of the starting position, every orbit 
tends to a periodic motion, with eventually player A copying the previous move of player B, 
while player B is aiming to play one ahead from the previous move of player A. Plotting the 
index of the largest component of the utilities v A and v B ("the strategy" of player A resp. B) 
as a function of time we obtain the repeating pattern (repeating in 6 steps): 

time duration 0.33123 0.31767 0.33123 0.31767 0.33123 0.31767 

strategy player A 1 2 2 3 3 1 

strategy player B 2 2 3 3 1 1 

That is, during 0.33123 time units players A, B prefer strategies 1, 2 respectively, then for 0.31767 
time units they prefer 2, 2 and so on. 

For 13 = a := [\fl - l)/2 w 0.618 (ignoring usual convention we shall call this number 
the golden mean), the game is equivalent to a zero-sum game 4 : rescaling B to B = a(B — 1) 
gives A + B = 0. So all orbits tend to the equilibrium E = (E A ,E B ) with E A = (A B ) T = 
(1/3, 1/3, 1/3). However, the players will not choose strategies cyclically (this follows of course 
from Krishna & Sjostrom (1998)): one gets sequences such as 

strategy player A 1223331122332112211223322333223311221132233223332233111223311 
strategy player B 2233211223311122332233223321133112233222332233211331132233112 

Indeed, as will be shown in Sparrow & van Strien (2006), the convergence happens in a chaotic 
fashion and can be projected into one which is 'area preserving'. 

For (3 ~ 1 the players' game also tends to a periodic pattern, but now both player A and B 
aim to play one ahead from the other player, and therefore now infinitely often repeat (again 
repeating in 6 steps): 

time duration 0.12060 0.39493 0.12060 0.39493 0.12060 0.39492 

strategy player A 1 1 3 3 2 2 

strategy player B 3 2 2 1 1 3 

We call this the anti-Shapley pattern. The main point of this paper is to show how this change of 
behaviour occurs. It turns out that there are two critical parameters, a the golden mean sa 0.618 
and t w 0.915 such that 

• for (3 G (— 1,(t) the players typically end up repeating Shapley's pattern; in fact, for [3 <G 
(—1,0] regardless of the initial positions, the behaviour tends to a periodic one (for (3 > 
this is not true: then there are orbits which tend to the interior equilibrium and/or other 
periodic orbits). 

• for (3 <G (a, t) the players become extremely indecisive and erratic (and the moves become 
chaotic), while 

3 <t is the smaller of the two roots of <r 2 + cr — 1 = 0. 
4 We would like to thank the referee for pointing this out. 
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• for ft G (t, 1) the players typically end up playing the anti-Shapley pattern. 

For example, when (3 = 0.75 one sees erratic behaviour such as 

strat. A 331122233112211322333223322133113331133113311331133211221122 
strat. B 211221331122332223321133113331133211221122112211221112211221 

In this paper we shall describe the bifurcations that the periodic orbits undergo as (3 varies; in 
a sequel to this paper (Sparrow & van Strien (2006)), the nature of the chaotic motion will be 
described in more detail (for example, that even for (3 G (0, cr), there are many (exceptional) 
orbits that are attracted to the equilibrium point). 

Of course, when one or more of the players is indifferent between two strategies their dynamics 
is not uniquely determined. However, as we shall show in Corollary 2.1 (in Subsection 2.1) under 
a mild assumption only exceptional orbits of the play are affected by this ambiguity. In particular, 
the orbits discussed above are uniquely determined. 

2 Continuous Time Fictitious Play: the Generic Case 

We study the following version of continuous time fictitious play, which is convenient for calcu- 
lation, for numerical simulation, and for extracting the geometrical properties of the dynamics. 
It is equivalent to other formulations in the literature. Two players A and B, both having n 
strategies to choose from, have nx n pay-off matrices A = (a^) and B = (bij), 1 < i,j < n 
respectively. (The context will always make it clear whether we are talking about Players A or 
B or about the matrices A and B). We will assume throughout this paper that the matrices A 
and B arc both non-singular (i.e. have non-zero determinant). We denote the set of probability 
vectors for players A and B by S^, X B C R™. (Though both simplices are the same subset 
of R n , the subscripts are useful when we wish to make it clear which players probabilities we 
are discussing). At some time to G R the state of the game is given by a pair of probability 
vectors, (p A (to), p B (to)) G X := X^ x X#, representing the current (usually mixed) strategies of 
the two players. It is convenient to define p A and p B so that they are row and column vectors 
respectively. 

The dynamics of the system is determined as follows. At time to, each player computes a best 
response BRa{p b ) G XU and BRs(p A ) G X# to the other player's current strategy. If we denote 
the set of pure strategies for the two players by P A G Y,a and P B G X#, 1 < i,j,< n, then 
generically 5 player A's best response BRa is a pure strategy P A G XU, for some i G {1, . . . , n} 
chosen so that the i-th coordinate of the vector 

v A (t ) =Ap B (t Q ) 

is larger than the other coordinates (the non-generic case, when several coordinates of the vector 
v A (to) are equal will be treated later on). Similarly, player B's best response BRb is generically 
a pure strategy P B G X^, j G {1, . . . , n} where j is chosen so that the j-th coordinate of the 
vector 

v B (t )=p A (t Q )B 

5 In this paper the word 'generic' means that one is not in a codimension-one subspacc 
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is larger than any of the other coordinates. In cases where players are indifferent between two or 
more strategies (the non-generic case) , any convex combination of the relevant pure strategies is 
an equally good best response, so 

BR a ■ — > Y, A and BRb ■ Ta — ► 

arc in fact set-valued maps. This means equation (2.2) below actually defines a differential 
inclusion rather than a differential equation. 6 

Once best responses are selected, the dynamics is determined by the piecewise linear equa- 
tions: 

dp A /dt = BR A (p B )-p A 

dp B /dt =BR B (p A )-p B [Z - Z) 

so that each player's tendency is to adjust their strategy in a straight line from their current 
strategy towards their current best response. 7 In the GENERIC case as discussed above, player 
A continuously adjusts her probability vector p A (t) from its current position p A (to) in a straight 
line towards P A , and player B similarly adjusts p B (t) to move in a straight line towards P B , 
so that for times greater than t 0} and for so long as strategies P A and P B remain unique best 
responses, we may write, after a reparameterisation of time, the solution of (2.2) becomes 

P A (t + s) = p A (t )(l-s) + s-P A , 
P B (to + s) =p B (t Q )(l- S )+s-P B , 

for s e [0,1]. Note that we have parameterised time so that both players move towards their 
best response strategies at a uniform speed, such that if their best responses did not change they 
would arrive at (P A ,P B ) e <9£ at time t = to + 1, at which point the dynamics would halt. 
The parameterisation is chosen for ease of calculation and computation, and does not affect the 
geometry of the dynamics which is our chief concern in this paper; it would be straightforward to 
use the original time g = — ln(l — s) if an exponential approach to (P A , P B ) in infinite time were 
preferred (the speed of a solution of (2.2) is proportional to the distance to the target point). In 
any case, since orbits consist of pieces of line segments, numerical simulations can be done easily 
and with arbitrary accuracy. 

Equations (2.3) therefore determine the dynamics up until such time as one or other (or both) 
players become indifferent between two (or more) pure strategies. 

Let us first describe the sets where the best responses are constant. We will be particularly 
interested in games where there is an interior equilibrium] that is a point E = (E A ,E B ) in 
the interior of T. A x Tib where both players arc indifferent between all n strategies, so that all 
components of AE B are equal, and all components of E A B are equal. The following lemma is 
entirely straightforward. 

6 We will deal with this apparent complication in sections 2.1 and 2.2 below. Our approach will be to consider 
the flow where best responses are uniquely defined and to investigate the extent to which this flow extends in a 
unique and continuous way onto the parts of phase space where best responses are not unique; for the moment 
readers unfamiliar with the idea of differential inclusion will lose very little by continuing to think of (2.2) as a 
differential equation. 

7 The best response correspondences p B i— > BRa{p b ) and p A >— > BRb(p A ) arc upper scmicontinous with values 
closed, convex sets. It follows from Aubin & Cellina (1984, Chapter 2.1) that through each initial value there 
exists at least one solution which is Lipschitz continuous and defined for all positive time. In a sequel to this 
paper we will show that, under mild regularity conditions, any solution is in fact piecewise linear. 
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Lemma 2.1 (The complement of the indifferent sets). Assume that A and B are n x n non- 
singular matrices. Then the set where one player is indifferent between two given strategies ( and 
so where she has a choice of best responses) forms a codimension-one hyperplane and for each 
1 < i,j < n the set 

Sn = {(P A ,P B ) ; BR A {p B ) = P A and BR B {p A ) = P B } 

is convex. 

If there exists an equilibrium E = (E A 1 E B ) in the interior o/S, then all n 2 regions Sij are 
non-empty, and their closures meet at (E A ,E B ). 

Figure 1 shows a 3 x 3 example in which the two simplices T,a and £ b are each two dimensional 
triangles, with vertices representing the pure strategies of the two players as labelled, and the 
regions Sij with different aims marked in the figure. The state of the system at a particular 
time, (p A (t) , p B (t)) is represented by a pair of points, p A and p B , one in each simplex. The local 
direction of motion is towards the vertex determined by the location of the other players position 
in her simplex, as illustrated. Motion will continue in a straight line until one or other player 
hits an indifference line (see caption). 




Figure 1: An example in which the simplices £a and £b the regions where players have a specific 
preference are as shown, separated by dashed lines which show the indifference sets Z B C Sa where 
player B is indifferent between two or more strategies, and Z A cSs where player A is indifferent between 
two or more strategies. Starting from an initial condition (p A ,p B ) (marked with the * symbols), player 
A moves towards P A , while Player B starts by moving towards P B and changes direction towards P B 
as p A crosses Z B . The sets Sij € R 4 are the sets formed as products of two kite-shaped regions where 
each player has a strict preference. 

2.1 Uniqueness and continuity of dynamics where at most one player 
is indifferent 

We now consider, for general matrices A and B, the hyperplanes where one or other player is 
indifferent between two strategies. 
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Let us denote the set where player A is indifferent between strategies P A and P A by Z A ^ C S_b- 
We define Zf d C E A similarly. If we set Z A = UijZ^, then E A x Z' 4 is the set where player A 
does not have a unique best response. We define the set Z B similarly. Outside the set 

Z = (H A x Z A ) (J (Z B x E B ) 

where at least one of the players is indifferent between two or more strategies the flow is uniquely 
defined by (2.2) and is clearly continuous. If we define 

Z* = Z B x Z A C E A x E B , (2.4) 

which is the subset of Z where both players are indifferent between at least two strategies, then 
the next lemma says that the uniqueness and continuity extends to the complement of Z* pro- 
vided matrices A and B satisfy the condition (2.5) in the lemma. (Note that this transversality 
condition is satisfied for the Shapley example from above if and only if [3 ^ 0.) 

Proposition 2.1 (Transversality condition implies local continuity and uniqueness outside the 
set where both players are indifferent). The motion defined by equations (2.2) forms a continuous 
flow on (E^ x Eb) \ Z* provided the following condition is met: for any point (p A ,p B ) ^ Z* 
with, say, p A € Z B , p B Z A and strategy P A preferable for player A, 

the vector P A is not parallel to the plane Z B C E^ at p A (2-5) 

(and similarly, in the case when the role of p A and p B is reversed). 8 

Proof. The transversality condition (2.5) ensures that a point is moved immediately off the 
codimension-one plane Z B x E B so that player B has a unique best response the moment the 
motion begins. □ 

Since Z* has zero Lebesgue measure and consists of codimension two planes, this shows 

Corollary 2.1. Assume the tranversality condition (2.5) holds. Then there exists a set X C 
Y*a x Es of initial conditions such that 

• for each starting point in X the flow defined by (2.2) is continuous and unique for all t > 0. 

• X has full Lebesgue measure and is open and dense. 

2.2 Local dynamics where both players are indifferent: outside sets of 
codimension three 

Even though the previous corollary shows that most starting positions never enter the set Z* 
where both players are indifferent, it is still important to analyse the dynamics near this set. As 

8 The linear space of vectors where the I and i'-th options are indifferent to player B consists of the vectors in 
R m which are orthogonal to the difference of the Z-th and i'-th column vector of the matrix B. That the fc-unit 
unit vector in R m is not contained in this, means therefore that b^i ^ b kt r . So the above transversality condition 
holds if within each column vector of A, the coefficients are all distinct (i.e. a;j ^ a^j when i ^ i'), and similarly 
bkl 7^ frfci' when I ^ V . Clearly the set of matrices for which this holds is open and dense (and of full Lebesgue 
measure) . 
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it turns out, in many places the flow outside Z* extends continuously to Z* . This suggests a 
natural choice of the non- uniquely defined dynamics within Z* . Much more importantly - even if 
we are not interested in orbits that lie on Z* and do not want to make a choice for the dynamics 
on this set - the continuous extension dynamics on Z* is an "organising principle" for nearby 
orbits, as we shall see below. This continuous extension dynamics makes that all orbits near Z* 
have consistent dynamics. This point of view puts us in a classical dynamical systems framework 
and allows us, for example, to describe bifurcations of the flow (i.e. changes in dynamics when 
the parameter (3 changes) for orbits near Z* (as we shall see in the Section 3). 

Before going into the general case in the next subsection, let us consider the codimension-two 
case. To do this, let us first review the dynamics in the 2x2 case (this of course is well known, 
but it is useful to be explicit here as we shall use the classification later on). In this case the 
probability simpliccs and T,b in R 2 can both be identified with [0, 1] and so x Eg with 
the square [0, 1] x [0, 1]. Player A moves left or right, depending on the position p B of player B, 
i.e. depending whether (p A ,p B ) is above or below a certain horizontal line, and similarly player 
B moves up or down depending on whether {p A ,p B ) is to the left or right of some vertical line. 
Drawn in Figure 2 are a few orbits of the system in various cases and • marks the aims (0, 0), 
(1, 0), (0, 1) and (1, 1). The dotted lines denote where one player in indifferent to two strategies. 
From left to right cases (1), (2), (3), (2) and (3). In cases (2) the orbit spirals towards the interior 
point, while in case (3) the interior point acts as a saddle. 9 




Figure 2: The motion in various cases for 2 x 2 games. 



Lemma 2.2 (2x2 case). Assume that both players have two strategies and that the corresponding 
2x2 matrices are non-singular. Then there are three possibilities: 

1. there is no interior equilibrium in xS^ and all orbits tend to a unique singular point on 
the boundary (this occurs if Z A and or Z B are empty, and is depicted in (1) in Figure 2); 

2. there is an interior equilibrium E which is stable, orbits spiral towards it and the flow 
extends continuously to E (this is depicted in (2) in Figure 2); 

3. there is an interior equilibrium E which is a saddle point and the flow cannot be extended 
in a continuous way to E and there is genuine non-uniqueness in the flow near E (this is 
depicted in (3) in Figure 2). 



9 In case (3), the orbit through a point in the stable manifold enters the equilibrium point in finite time! 
Moreover, through the interior equilibrium point E there are an uncountable set of solutions; one of these remains 
in E for all time (at least locally), and the others stay in E for some (possibly small) finite time which can be 
chosen freely, and then fall off in the unstable direction. 
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As we shall see in the next section, in case (2) the solution tends to E as t — > oo, spiralling 
faster and faster around E in a very precise way. 10 This means that we can think of the flow at 
E as just stationary there; the flow is continuous (and each solution is unique). 

We now consider the behaviour of the system when both players are indifferent between two 
or more strategies (but now for general A and B) . This occurs on the set Z* = Z B x Z A defined 
above. Since we are interested in typical trajectories of the system, and typical trajectories will 
not actually hit Z* , we are particularly concerned with the implications of this for trajectories 
near to but not on Z* . This contrasts with the case of the codimension one sets considered above; 
typical trajectories do hit those. We will see, though, that we can in many circumstances define 
a unique flow for points in Z* as well. 

Consider the subset Z' kl { - of Z kl x Z A - C Z* where player A is indifferent only between strate- 
gies i and j, and player B is indifferent only between strategies k and I. Since we assumed that 
A and B are non-singular, this is a codimension- two plane in XU x Define a neighbourhood 
of Z' kl ij by 

N = {( P A ,P B ) ; BR A (p B ) C {i,j} and BR B {p A ) C {k,l}} . (2.6) 

In the set N, player A prefers strategies i and j to all other pure strategies, and player B 
prefers strategies k and I. In N \ Z', because of Proposition 2.f above, equation (2.2) can be 
justifiably interpreted as a differential equation with unique solutions (rather than a differential 
inclusion with possibly non-unique solutions) with the dynamics consisting of a linear decline in 
all components p^, m ^ i, j and p B ,n^k,l while the behaviour of the other two components of 
p A and p B depends entirely on the 2x2 submatrices of A and B: 

A[k,l}=( ak ' k ak < 1 ) B[i,j} = ( ^ ^ V 

These matrices describe a game on the two-dimensional space spanned by [P A , P B ] x [P B , P B ] 
(where [P A , P A ] stands for the line segment \P A + (f - \)P A C S A , < A < f ). It follows 11 that 
the dynamics in N is the product of the dynamics in this two-dimensional space (on which the 
dynamics is as in Figure 2) and a linear decline along the codimension-two plane in Z' . Together 
with Lemma 2.1 this gives: 

Proposition 2.2 (Continuity and uniqueness outside sets of codimension three). Assume that A 
and B have maximal rank and that the transversality condition (2.5) is satisfied. Then the flow 
defined on the sets Sij has a unique continuous extension everywhere except (possibly) where: 

• one player is indifferent to at least three strategies and the other to at least two; 

• both players are indifferent to precisely two strategies and along this set the dynamics is as 
in case (3) described above (in Lemma 2.2). 

In the neighbourhood N of Z' kl - defined in (2.6) the flow decomposes into a flow governed by a 
two by two game, and a constant flow in the remaining direction(s) and there are three cases: 

10 In this case the orbit through any point outside the equilibrium never hits the equilibrium point. 

11 A similar decomposition is also used in Hofbauer (1995, page 6 and Appendix A). Amongst other things, 
Hofbauer shows in this interesting paper that the dynamics starting at a point x £ X can proceed in a 
straight line in the direction b £ T,a x Eb iff b is a Nash equilibrium of the restricted game at x. Here we use the 
same approach to investigate uniqueness. In a sequel to this paper, we will push this point of view further. 
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1. trajectories cross either or Zfj and locally the flow is the product of a constant flow 
perpendicular to the plane and a flow as in (1) in Figure 2; 

2. trajectories in N spiral towards E (i.e. towards Z' ) along a cone over a quadrangle as in 
Figure 3, and move closer and closer to the unique trajectory in Z' that remains in Z' - 
this trajectory is therefore the unique continuous extension of the flow in N onto Z' - and 
locally the two-dimensional flow is as in (2) in Figure 2; 

3. Z' acts as a saddle and generic trajectories leave N moving away from Z' in one of two 
opposite directions and locally the two-dimensional flow is as in (3) in Figure 2. 

Proof. Only the 2nd case needs some explanation. Let S be the two dimensional plane spanned 
by the four targets (Pf, P B ), s e {i,j} and t € {k, I}. Moreover let V C S be a quadrangle with 
sides parallel to the lines connecting the equilibrium E e S to (Pf,Pf), s e {i, j} and t G {k, 1} 
(in such a way that in the region where the players aim for (Pf, P t s ) one takes a line parallel to 
the line connecting ) to (Pf,P^)). Now taking any quadrangle V r parallel to V, and the cone 
through the equilibrium E G S over V r . Clearly orbits that start in this cone, will stay in it. □ 




Figure 3: The targets (P s , P t s ), s € t € {k, 1} in the plane S are drawn marked as •. The dashed 

line corresponds to Z' . A quadrangle T r in S is also drawn and also another quadrangle T' r which is 
in a plane S q (through q 6 Z') which is parallel to S is drawn. The cone of the quadrangle over the 
equilibrium point E is invariant. Orbits spiral along the cone towards E, always aiming for one of the 
points marked as •. Note that the motion on the cone is not close to the direction of the axis Z' . 

In case (1) though both players have a choice of best response at (p A ,p B ), there is no choice 
of best responses that keeps the trajectory inside Z*. This transversality ensures that the de- 
generacy occurs only at discrete times (i.e., 'momentarily' if we may put it like that), and there 
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is actually only one solution to the differential inclusion through (p A ,p B ). In case (2) there is 
a unique choice of best responses on Z* for both players such that the flow defined by equation 
(2.2) remains within Z*. Nearby solutions remain close to and spiral (locally) towards this so- 
lution, and so it is the continuous extension of the (unique) flow off Z*. Again, the apparent 
non- uniqueness is an artefact; we may say if we like that any attempt to depart from this solution 
in Z* is immediately thwarted by the neighbouring flow. Only in case (3) do we find genuine 
non-uniqueness of solutions to the differential inclusion. This amounts to the fact that there is a 
choice of best response for each player which keeps the flow within Z* , and all other choices move 
you immediately off Z* in one of two directions. Thus through (p A ,p B ) there are an uncountable 
set of solutions; one of these remains in Z* for all time (at least locally), and the others stay in 
Z* for some (small) finite time which can be chosen freely, and then fall off in one of the two 
directions (again chosen freely). 

3 Existence and stability of periodic orbits in the Shapley 
family 

From now we shall concentrate on a particular family example which includes a classical example 
by Shapley. These examples are given by matrices 

/ 1 (3 \ ( -(3 1 \ 

A = \ (3 1 B = -13 1 , (3.7) 
V (3 1 / V 1 0-/3/ 

When (3 — 0, this means that the utility of player A is equal to p B , so player A's will aim 
for the largest component of p B , while player B utility is (p A ,p A ,p A ) = p A B so B is aiming 
one ahead from the largest component of p A . So for (3 = 0, it is like the paper, scissor, rock 
game. Our interest is to understand how the behaviour in this game changes when (3 changes. In 
Subsection 3.5 we show that the player's actions will become extremely erratic for certain values 
of (3. 

Figure 4 shows pictures of the phase space in the cases — 1 < (3 < and < (3 < 1, marking 
the lines where the players are indifferent by dashed lines. (In Figure 1 the phase space was 
drawn for (3 = 0.) Note that for all values of (3 both players are indifferent between all three 
strategics at the point E = (E A ,E B ) where E A = (E B ) T = (1/3,1/3,1/3). Note that when 
(3 7^ 0, for each p A e Z B C XU, each aim vector p A P A is transversal to Z B , and similarly for 
p B E Z A C S B . So the transversality condition of 2.5 holds for (3 ^ 0. 

3.1 Continuity and uniqueness of the flow 

We now apply the results of the previous section to the family of examples defined by matrices 
(3.7). An essential difference between the cases (3 < and (3 > can be understood in terms of 
that analysis. 

Proposition 3.1 (Continuity and uniqueness). When (3 € (0, 1) the unique flow on the comple- 
ment of the indifference set Z extends to a flow on Z which is everywhere continuous and unique 
except at E = (E A ,E B ). When (3 £ (-1,0], the unique flow on the complement of Z does not 
extend continuously to some codimension two surfaces in Z* . In more detail: 
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Figure 4: The simplices Ea and Eb for /3 < (above) and (5 > (below). For /3 £ [—1, 1], the sets where 
Player A is indifferent between two strategies are line segments Z(j in E_g connecting the equilibrium 
point£ A = (1/3,1/3,1/3) to the points ,-^(1,1-/3,0) -r^j (1 -/?, 0, 1) and (0,1,1-/3) (marked 
with •). Player B is indifferent between two strategies along line segments Zfj in Ea connecting the 
midpoint (1/3, 1/3, 1/3) to the points (1 + P, 1, 0), p^y (F 0,1 + /?) and ^aj(°> 1 + Z 3 . !) (marked 

with *). For (3 = these points lie on the middle of the sides of the triangles OTja and <9Eb (as in Figure 
1). Increasing f3 £ [—1, 1] rotates the lines anticlockwise (so the end point moves for example from 
towards P\). Note that these indifference lines in the simplices T,a and Sb do not coincide (despite the 
appearance of the figure); for example, when /? = 1 then in Eb the indifference lines Z^j go through the 
corners, but the lines Zfj in Ea do not. 

• If (3 £ (—1,0) then the transversality condition from Proposition 2.1 holds for all codimen- 
sion one indifference points in Z. All the codimension two points in Z* are in cases (1) 
or (3), and the flow off Z extends continuously to those points that are in case (1). The 
differential inclusion (2.2) defines a flow that is unique except at those points, where the 
flow is discontinuous. 

• If [3 = then the transversality condition from Proposition 2.1 fails to hold in several 
codimension- one sets (for example, the set Z^ 2 x {z £ T,b ', BRa{p b ) — Pi'})- 

• If (3 € (0, 1) then the transversality condition from Proposition 2.1 holds on all codimension 
one indifference sets, and all the codimension two points in Z* are in case (1) or (2). In 
particular, the flow off Z extends continuously to all points except E = (E A , E B ), and so 
the differential inclusion (2.2) defines a unique and continuous flow everywhere except at 
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the interior equilibrium E. 

Proof. This can be seen just by looking at the possible 2x2 games that can be obtained from 
submatrices of A and B and applying the results from the previous section. (It is also possible 
to convince oneself of the result graphically by comparing the disposition of lines Z i ' and 
points and Pf in Figure 1 and 4.) When (3 < case (3) holds, for example, in Z± 2 X Z. A 



2,3- 



When (3 > there are points in Z* , for example on the set Zf 2 x Z\i where case (2) arises and 
trajectories locally spiral around and tend towards Z* (until they meet another set in Z*). On 
the other hand orbits cross Zf 2 x ^23 (which is in case (1) transversally when j3 > 0. □ 

We will see later that when (3 > there is genuine non-uniqueness of the flow at E. The 
spiralling behaviour that can occur near Z* for [3 > contributes significantly to the extra 
richness of behaviour in f3 > 0. The subset J C Z* 12 on which we have this behaviour consists 
of the six pieces of Z* : 

J = (Z* 2 x Zi x ) U (Z* 2 x Z? a ) (J(Z 2 B 3 x Z A 2 ) U (Z* 2 x Z 2 A 3 ) |J 



The set consisting of the remaining three pieces of Z*: 

T = (Z* x Z^) |J(Z 2 B 3 x ^) (J(Z 3 B 1 x Z^ 2 ) 
contains the codimension two points where the flow is transversal (case (1)). 



(3.8) 
(3.9) 



3.2 Existence and stability of a clockwise Shapley periodic orbit for 

pe(-i,a) 

Let a be the golden mean. We shall first show the existence of a stable periodic orbit for 
(3 € (— l,o-). We call this the Shapley orbit, as it is the continuation of the orbit described by 
Shapley (1964) for the case (3 = 0, or the clockwise orbit, as the projections of the orbit into 
the spaces and Y,b rotate clockwise about the equilibrium, see the figure below. In the next 
subsection we shall also show that this orbit is globally attracting when (3 < 0. For (3 = orbits 
spiral away from the equilibrium points towards a periodic orbit (which consists in 4 dimensional 
phase space x of 6 pieces of straight lines). 



P= 





E A simplex 



E R simplex 



12 We could take J to stand for "jitter", since as nearby trajectories spiral around J, both players jitter back 
and forth between strategics. 



13 




Theorem 3.1 (Existence of the Shapley periodic orbit). For (3 E (— l,a) there exists a periodic 
orbit in which player 1 and 2 infinitely repeat the strategies 

time 1 2 3 4 5 6 

strategy player A 1 2 2 3 3 1 
strategy player B 2 2 3 3 1 1 

(So player A copies B and player B tries to play one ahead of A.) This is a continuous defor- 
mation of the orbit that Shapley found for [3 — 0, and as (3 \ a this orbit shrinks in diameter to 
zero and tends to the point (E A , E B ). For [3 G (a, 1) this periodic orbit no longer exists. 

The proof of this Theorem is given in Section 5.1 (in the Appendix). Figure 5 (and later 
figures up to Figure 7) allow us to illustrate schematically the relationship between regions $y 
where player A prefers strategy i and Player B prefers strategy j, 1 < i,j < 3. The 9 squares in 
these diagrams each represent one such region, as labelled. We think of the 3x3 checker board as 
a torus, so that the top and bottom are identified, as are the left and right sides (so if we go for a 
walk on the checkerboard and wander off the top, we return with the same horizontal coordinate 
at the bottom, and similarly if we wander off the left of the diagram we return with the same 
vertical coordinate on the right). With this convention, the diagram illustrates the adjacency 
of regions; if just one player changes her preferred strategy, we cross a horizontal or vertical 
line (representing a codimcnsion-one set) to get to the new region. If we wish both players to 
change their preferred strategies at the same instant, we must cross diagonally through a corner 
(representing a codimension-two set) into a new region. Higher codimension points, such as the 
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internal equilibrium (E A ,E B ) where all regions meet, are not represented. Note also that each 
region is actually 4-dimensional, rather than 2 as represented in the diagram, and so if we wish 
to project trajectories of the system onto the diagram we may find that they cross; certainly we 
do not want to claim there is any sort of unique flow determined on the checkerboard. 

The usefulness of these diagrams is that we may add arrows showing which transitions are 
allowed, which we will use in the proof of the theorem. Thus, for example, if we imagine that 
we begin in the top left-hand region where A — > Pf 4 and B — > P B then inspection of Figure 1 
for (3 = shows that the only region we can move to (and indeed will move to in due course) 
is the region where A — > P A and B — > P B , as represented by the arrow between these two 
regions in Figure 5. The other arrows are computed similarly. In regions such as A — > Pf 4 
and B — > Pf there is a codimension-one set such that trajectories started on this set will reach 
indifference lines in and S b simultaneously. This set is represented by the dotted line in the 
appropriate square in Figure 5, and similarly in other squares where this occurs. Finally, some of 
the codimcnsion one indifference surfaces are represented by dashed lines in the (3 = case. This 
is rather special, and is where the transversality condition of Proposition 2.1 fails; orbits move 
off in different directions on opposite sides of these lines. Figure 6 for (3 < is rather similar, 
and the extra dotted lines are explained in the caption. 



B 



XU-plane 
(p^-coordinates) 



B 



(i.e., ^-coordinates) 
A -» Pf 4 A 



P 2 A A 



pA 



pB 



I* 1 — 



B^P* 



+ 



_ _ 1 _ 



_ _ 1 



3 



4 



(3 = 



Figure 5: A diagram showing allowed transitions between regions of pure preference for (3 = 0. See the 
text for explanations of dotted and dashed lines. 



Let us call the discontinuity set the subset of Z* for which type (1) behaviour occurs. It has 
(of course) measure zero. 

Theorem 3.2 (The Shapley orbit is globally attracting for (3 < 0). For (3 < 0, the flow has a 
unique closed orbit (the Shapley orbit); the flow of all initial conditions which do not enter the 
discontinuity set tends to this closed orbit. 

Proof. When (3 < 0, then the only regions which can be entered are those marked with a central 
integer label. So any trajectory which does not enter the discontinuity set, eventually enters one 
of these regions. (These orbits form a set of zero Lebesgue measure.) From the diagram it is clear 
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X_b (i-e., ^-coordinates) 



P A A 



P A A 



pA 
P3 



B 



S^-planc 
(p A -coordinates) 



B 



pB_ 



P B - 



B -» P( 



2 — 



+- 

6 - 



+ 



4 

- 5 

-f- 



/3 < 



Figure 6: The transition diagram of the flow for j3 < 0. The dotted lines again indicate the codimension- 
one sets where p A and p B leave the relevant regions simultaneously. The points where these dotted 
lines meet, correspond to a set of initial conditions where the trajectory exits the region by hitting 
P 4 = P s = (1/3, 1/3, 1/3). 



that from then on the orbit cyclically visits the regions 1-6. Now consider, for example, the part 
S\ of the space Z A 2 where player A is indifferent between strategies P 2 A and P A , and player B 
strictly prefers strategy P 2 B (this corresponds in the diagram to the line dividing regions labelled 
1 and 2). Similarly consider the region 5*2 where the orbits leave region labelled 2. Each of these 
sets Si is the product of a line segment and a kite-shaped quadrangle, with one boundary point 
the equilibrium point E. Now consider the map T which assigns to a point in S\ the first visit 
of the forward orbit through x to S^- Since all orbits in region 1 will aim for the same point and 
only exit via 5*2 (when (3 € (—1,0]), this map is continuous; moreover, it has the property that 
T(p A ,p B ) g interior(S 2 ) for each (p A ,p B ) g Si \ (Si n S2). In fact, T is a central projection. In 
other words, if we put linear coordinates on Si and S2 then T: Si — > S2 takes the form 

T(z) - J^j L ( z ) ( 3 - 10 ) 
where L : Si — > S2 and / : Si — ► R are affine transformations. So in linear coordinates, 

Tlx y z) — ( ai+ + ° iy + 0,2 + ^ 2X + ° 2y + ^ 2Z ° 3 + ^ 3X + ° 3V + ^ 3Z 
\ a + bx + cy + dz ' a + bx + cy + dz ' a + bx + cy + dz 

(These maps are explicitly calculated in the appendix; here we shall only use that it is of this 
form.) Note that compositions of such maps are again of this form. If 

Pi = -j-Li and P 2 = ]rL 2 
Ji .72 

are two such transformations and we take linear coordinates such that Pi(0) = (i.e. Li(0) = 0) 
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then the composition formula becomes particularly simple: 



T 2 oT 1 = — — 1 . L 2 oLi(z). (3.11) 
/2(Ti(z)) • /i(z) 

Note that because of cancellations f2{T\{z)) ■ fi(z) actually is an affine map in z. 

Now let P be the first return map to S\. As we observed P is a continuous central projection 
as above (it is composition of such maps), so of the form z ^ P(z) — j^L(z) with P(Si) C Si 

and, because T: Si — ► S2 maps all points except those in Si n S2 into interior^) (and similarly 
for the other transition maps), 

P(S7) C interior(Si) U E. (3.12) 

In addition, P has a fixed p in the interior of Si corresponding to the Shapley periodic orbit. Let 
us show that lim„^oo P n (x) — > p for each x £ Si \ E, where P n is the n-th iterate of P. To see 
this, notice that P sends lines to lines and restricted to each line, T is a Moebius transformation, 
i.e. of the form t 1— > ^q^ff (where we take affine coordinates on the lines). Let us first consider 
the line Zo through p and £7, and let b be so that Zo n Si = 6] . This line Zo is mapped into itself 
because p,E € Zo are both fixed points. P : Z — > Z is a Moebius transformation which sends the 
line segment [E,b] — Z n Si continuously into [a, 6) (this is because of (3.12)) and having fixed 
point p e (E, b). It follows from this that for any x € (E, b], lim^oo P n (x) — > p. Now consider 
any other eigcndircction Z through x. Because of (3.12), P maps lC\S\ continuously strictly inside 
itself, and since the restriction of P to this line is a Moebius transformation, again all points in 
Z n Si converge to p. It P has a complex eigenvalue then the argument is similar. 

To consider any other point x, take the projective space P of lines through p in Si. The 
return map P assigns to each line x £ P a new line P(x) € P. The eigenvalues of the linear 
transformation L correspond to fixed points of P, and for each line Z through x in Si, P n (l) 
tends to an eigenspace of the linearisation of P at x. Combining this with the above, it follows 
that for each xeSi\E, P n {x) -» p. 

It follows that for the original flow all trajectories which do not enter the discontinuity set 
are attracted to the periodic Shapley orbit. □ 

This theorem completes our investigation of (3 < 0. 



3.3 Existence and Stability of a Shapley and also an anti-Shapley pe- 
riodic orbit for (3 > 

Let us now consider orbits when [3 G (0,1). The transition diagram for this case is given in 
Figure 7 and it is immediately clear that there are now many more possible routes that may 
be taken from region to region. We stress that the existence of a possible route allowed by the 
arrows, does not necessarily imply the existence of a corresponding trajectory in the flow; we 
will need to perform some calculations to see which routes are actually realised for any given 
value of (3. In this figure the dashed lines represent codimension-one sets of points from which 
trajectories tend, for one or other player, to E A or E B . The sequence of regions visited will be 
different depending which side of these sets you begin. [We should note that the dashed curves 
do not move entirely across the squares as (3 — > 1 (because the points in Figure 4 marked with • 
in T,b still have not reached the corners of when (3 — 1).] 
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B^Pi 



B -» Pf — 



\ 



+ 



\ 



\ 



1>(3>0 



Figure 7: The transition diagram of the flow for (3 G (0, 1). Boxes again represent regions Sij, and arrows 
indicate possible transitions between regions. Routes may now spiral around near the 'corners' marked 
with *; these 6 corners represent codimension-two points on the six pieces of J. The 3 corners marked z 
represent codimension-two points on the 3 pieces of Z* \ J and trajectories cross these sets transversally; 
there is no spiralling. The numbers 1 — 6 in the regions show a possible route for an anticlockwise 
periodic orbit. Notice that the route for the clockwise Shapley orbit is still also allowed. The dashed 
lines correspond to preimages of points (E A ,p B ) or (p A ,E B ), whereas the dotted lines correspond to 
preimages of Z* \ J, but where the orbits will cross transversally as in case (1) above. When /3 = 1 the 
situation remains almost the same except that it is no longer possible to have two types of exits from the 
regions with the dotted lines (no exit is possible in the horizontal direction in the three relevant boxes, 
but this restriction does not prevent the continued existence of the anticlockwise route marked). 



We show in the Appendix, by direct calculation, that precisely one of the anticlockwise or 
clockwise orbits mentioned in the figure caption exists when (3 ^ a, where a = (— 1 + \/5)/2, 
the golden mean. In particular, for (3 <G [—1, er) there is an attracting orbit which goes round the 
midpoint of £a and Es in an clockwise fashion (this is a continuous deformation of Shapley's 
periodic orbit for (3 — 0, which we have already studied in the previous section for [3 < 0), 
whereas for (3 G (er, 1) there is a different periodic orbit which goes around these points in a 
anticlockwise fashion. It is attracting for some interval of (3 values to be described further below, 
and is otherwise of saddle-type. 

Theorem 3.3 (Existence and stability of the Shapley and anti-Shapley periodic orbits). For 

[3 G [0, a) there exists the periodic orbit mentioned already in Theorem 3.1, i.e. it visits, in 
the same order, the same regions as the Shapley orbit of Theorem 3.2. This is a continuous 
deformation of that orbit, and as (3 | a this orbit shrinks in diameter to zero and tends to the 
point (E A , E B ). This orbit is stable for all (3 G [0, a). 

For (3 e (a, 1) there exists a periodic orbit (anticlockwise) which visits the regions marked 1-6 
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in Figure 7 cyclically in order, i.e., follow the following pattern: 

time 1 2 3 4 5 6 

strategy player A 1 1 3 3 2 2 
strategy player B 3 2 2 1 1 3 

As (3 [ a this orbit shrinks in diameter to zero and tends to the point (E A , E B ). Such a periodic 
orbit does NOT exist when (3 < a. 

Shapley's (clockwise) periodic orbit is attracting for all (3 G [0,cr). The anti-Shapley (anti- 
clockwise) periodic orbit which exists for (3 G (a, 1) is attracting when [3 G (r, 1) and of saddle-type 
when (3 G (cr, r). Here t is the root of some specific expression and is approximately 0.915. 

We remark that it does not seem to be possible to prove the existence of the anticlockwise 
orbit, even when (3 = 1, by the same sort of technique as we used to prove the existence of the 
clockwise orbit in [3 < 0. This is because even when [3—1 the relevant version of Figure 7 
still allows many routes through the diagram; it was only because there was only a single route 
possible in the (3 < case that we were able to deduce that an orbit following that route must 
exist. Neither the Shapley nor the anti-Shapley orbits attract every starting position: as we show 
in the sequel to this paper Sparrow & van Strien (2006), there are an uncountable number of 
orbits which tend to the interior equilibrium point. The existence of one such orbit is shown in 
the next subsection. 

The proof of this Theorem is in the Appendix, as is a discussion of the bifurcation at (3 = r. 
3.4 Dynamics and a periodic orbit T on the set J 

We will now consider the flow on the set J (more precisely, the unique continuous extension of the 
flow to J) . The dynamics near J is very interesting because orbits can spiral for very long time- 
lengths near J, before moving away. Because on J at least one of the two players is indifferent 
between two strategies, nearby orbits correspond to behaviour where players frequently switch 
their preferred behaviour. 

We shall prove here that trajectories on J visit the six pieces cyclically (in the order in which 
they appear in the definition in equation (3.8)), and the details of the behaviour are given in the 
following proposition (see Figure 8). 

Theorem 3.4 (A third periodic which is created by a Hopf-like bifurcation). The system has a 
Hopf-like bifurcation when (3 — cr on the topological two-dimensional manifold J. Trajectories on 
J spiral towards (E A ,E B ) when (3 G (0, cr] while for (3 G (cr, 1) all trajectories in J (except the 
one stationary at (E A ,E B )) tend to a periodic orbit T in J . 

We denote the endpoints of T along the pieces of J by f A - and f B t , as in Figure 8. Then for 
(3 G (cr, 1) one has the following formulas for the 'diameter' of the periodic orbit T: 

\E£-.m \f*-m A \ _ P 2 +f3-l 

\Q%-f&\ 1^1-^1 2 + 1 

and 

\r%-e%\ \Qti-m (i+/?) 2 ■ 
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Note that these expressions are both increasing in (3 € (a, 1) and are zero when (3 — a (but unlike 
the usual H op f -bifurcation the diameter increases roughly linearly when /3 — o->0is small). They 
achieve their maximum 1/3, 1/4 both at [3 = 1. 

The proof of this Theorem is a tedious calculation and can be found in the appendix. Note 
that it is also argued there that trajectories spiralling into (E A ,E B ) on J for (3 < a, or out of it 
for (3 > a, do so in finite time. There is genuine non-uniqueness of the flow at (E A ,E B ) reflecting 
the fact that it is the only point (for (3 > 0) at which we cannot continuously extend the flow. 




Figure 8: The simplices Ea and Eg. Various points on the sets Z A and Z B (and their extensions) are 
drawn. R A A = 2^-3(1 + P, 1, 0), Q A 2 = 1+2/3 (Pi ^ Pify (and similarly formulas for the other points by 
cyclically permuting coordinates). Similarly Rf 2 = 5^3 (1> 1 — 0,0) and Qf 3 = j^fP, 1,0). The points 
/ij) /h give the maximum extent of the periodic orbit on J (/3 > a only). Take (p A ,p B ) G (Z^2 x Z^\) 
as starting points. Then p A (t) moves along Z B 2 towards R A 2 and p B (t) moves towards the midpoint 
of E s along Z A 3 . Once p B (t) hits the midpoint of E s (provided p A (t) (1/3, 1/3, 1/3)) then the flow 
extends continuously and (p A (t) , p B (t)) enters Z B 2 x Z A 2 . Along this set p A (t) then moves back along 
Z B 2 towards the midpoint in E^ while p B {t) moves towards R B 2 . Next it enters the third part of J, and 
in this way the orbit spirals (cyclically) around the different legs of J. 



3.5 Dithering and chaotic behaviour near J 

It is not true that this orbit is attracting for orbits that start off J. Nonetheless, orbits of the 
system for [3 > a can spend long times near to this orbit J. The reason for this is that orbits can 
spiral along cones whose axes consist of the 6 legs of the periodic orbit on J in the way described 
in Figure 3. The next two figure below shows a portion of a numerically computed trajectory for 
(3 = 0.8. The orbit shown, starts near the centre, and during the time interval considered comes 
close to the periodic orbit J, while jittering a large number of times near this orbit. Notice that 
the short time intervals and the large number of time the players switch strategies. 
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In fact, as will be made precise in Sparrow & van Strien (2006), one has chaotic motion for 
/3 6 [a, t]. The figures below show orbits for j3 = 0.9, both showing erratic time-series and the 
attracting set projected on Y. A and S B . 
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4 Conclusion 

For j3 € (—1,0] players always asymptotically become periodic. When /3 6 (0, cr) the Shaplcy 
orbit is still attracting but not globally attracting: there are orbits as described in Theorem 3.4 
which tend to E. In a sequel to this paper we shall show that there are infinitely many such 
cone-like sets consisting of points which tend to E. (So the stable manifold of E is extremely 
complicated.) 

The numerical results showed that for (3 £ (cr, r) players act erratically and chaotically. 
In the sequel to this paper we shall give rigorous results which explain this behaviour. The 
mathematical methodology in this sequel paper will be more geometric (and will be based on 
first projecting the flow on a three-sphere and then by a study of the first return dynamics to a 
global two-dimensional section consisting of a disc bounded by the periodic orbit T). 

Finally, when /3 E (r, 1) orbits tend to a so-called anti-Shapley orbit. 

Of course most of the analysis we described also could be applied to other families (with less 
symmetry). We will go into this in the sequel to this paper. Finally, simulations and preliminary 
theoretical considerations indicate that for n x n games with n > 3 there are other routes to 
chaotic behaviour than those studied here. 

Acknowledgements. We are grateful to the anonymous referee and an associate editor for 
helpful comments and suggestions. 
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5 Appendix 

In this appendix we collect together a number of explicit calculations that are used in the main 
text. 

It is convenient to work in the space (v A ,v B ) where v A = Ap B and v B = p A B (so v A is 
a column vector, and v B is a row vector). From the description of the dynamics in the main 
text, when v A is the largest coordinate of v A , player A adjusts p A towards P A , which implies v B 
moves in a straight line towards Bi, the i th row of B. Similarly, if v B is the largest component of 
v B , v A moves towards Aj, the j th column of A. In both cases, our convention is that the motion 
occurs at a speed such that from a given initial condition, and assuming that i and j were to 
remain the maximal coordinates of v A and v B , the vectors would reach Bi and Aj in one unit of 
time. So, for < s < 1, provided the maximal coordinate of v A or v B does not change, we can 
write 

v A (t + s) =v A (t) + s(A j -v A (t)) 

v B (t + s) =v B {t) + s{B t -v B {t)) • { ° 6) 

Each time that the maximal coordinate of either v A or v B changes, we reset s in (5.13) to zero. 
In this appendix we calculate explicit orbits with particular properties, using these equations. 
We illustrate our calculations with figures that show the development of the components of v A 
and v B with time. Let a = (—1 + \/5)/2, the golden mean. 
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5.1 Existence of the clockwise Shapley orbit in j3 e (— l,cr) 
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Figure 9: Components of v A and v B plotted against time for one third of the symmetric Shapley 
(clockwise) periodic orbit which exists for f3 G [0, a). Since v B is maximal throughout, v A (t) moves 
towards A2 = (0,1, P) T ■ In the interval [0,ti] coordinate v A is maximal and v B (t) moves towards 
Bi = (— (3, 1, 0); in the interval \t\,t\ + tz], v A is maximal and v B (t) moves towards B2 = (0, — /3, 1). 

Let us compute the clockwise Shapley periodic orbit which, as we will see, exists for (3 € 
(— 1, a). The periodic orbit we are looking for is rotation symmetric, so we will only compute the 
part of the orbit where player B prefers strategy 2 (while player A prefers first strategy 1 and 
then strategy 2); see Figure 9. 

Let n — {n ll n2 1 nz) T and m — (mi, 7712,7713) be the initial values of v A and v B respectively. 
Then we require, as shown in the figure, that 



m > max(ri2,r} 3 ) and mi = m 2 > m 3 , 



(5.14) 



so that initially v A heads towards A 2 = (0, 1, [3) T and v B towards (— /?, 1, 0) = B\. The first time 
t\ that the orbit meets Z again is when the first and second coordinates of v A (t) = (1 — t)n T + 
t(0, 1, (5) T become equal which occurs when t = t\ > 0, 



h = 



ni — n 2 + 1 

(Of course, it is possible that we first get that ni = n 3 , but this does not occur because n\ = n 3 

ni— ri3+/3 



happens when t' = which is larger than t\ as (3 < 1.) The v A and v B values at t = t\ 



are 



v A (h) 



1 



77,1 

n 2 + (t7i - 77 2 ) 



(ni ~ n2 + 1) V 77 3 + (ni-n 2 )/3 



26 



and 

v B (ti) = — — (mi - 0(m - n 2 ), m 2 + (m - n 2 ), m 3 ) . 

(ni - n 2 + 1) 

Next we have that i 2 > is the smallest number so that the second and third coordinate of 
v B (h + t 2 ) = (1 - t 2 )v B (t 1 ) + t 2 (0, -/?, 1) are equal. This gives 

m 2 (ii) - m 3 (ii) (m 2 - m 3 ) + (ni - n 2 ) 

£ 2 = 



m 2 (ti) - m 3 (ti) + 1 + (3 (m 2 - m 3 ) + (ni - n 2 ) + (1 + /3)(1 + ni - n 2 ) 
which implies 

/ m(i + /?) \ 

1^*1+ 1 2 ) = T m(l+/3)+J , (5.15) 

V ((n 3 + (n 1 -n 2 )/3)(l + /?)+/3<5) / 

and 

« B (ti + t 2 ) = ((mi - p{nt - n 2 )), (m 2 + (m - n 2 )) - /W, m 3 + 5/(1 + /?)) (5.16) 



where S — (m 2 — m 3 ) + (ni — n 2 ) and X = (5 + (1 + /?)(1 + n\ — n 2 ). For a symmetric periodic 
orbit we have to satisfy 





v A {h+t 2 )=\ m and v B {t 1 +t 2 )=\ mi , (5.17) 



and the strict inequalities (5.14). Solving equations (5.15)-(5.17) by hand (or by a MAPLE 
worksheet 13 gives 

n x =v (5.18) 
-2/3 2 - 3 + 2 v0 3 - 3 v - 2 v + 3 v 2 + 3 v 2 2 + 3 v 2 

n2 = i+TT^ ' (5 ' 19) 

2/3+l + 2i//3 3 -j//3 2 -4j//3-3i/ + 3z/ 2 + 3z/ 2 /3 2 + 3^ 2 /3 

na = TTlsTW 2 ' (5 ' 20) 

m 2 = mi = 1 — v — v0 (5.21) 

m 3 = -0 + 2v0 - 1 + 2u, (5.22) 

where v satisfies fp{v) — with 

fp{Z) = (3(3 2 + 30 + 3)Z 3 + (20 3 - 2/3 2 - 5/3 - 4) Z 2 + (-f3 3 + 40 + 3)Z-l-0. (5.23) 

Note that for G (-1,1), //}(0) = -1-0 < and f (j {\) = 3 + 2 + + 1 > 0. For e (-1,1), 
the function i //3(i) is monotone because its derivative 

f'p(Z) = 3 (3/3 2 + 30 + 3) Z 2 + 2 (2/3 3 - 2/? 2 - 50 - 4) Z + (-0 3 + 4/3 + 3) (5.24) 

is a strictly postive quadratic function; this is because its discriminant is equal to 16/3 6 + 4/3 5 — 
28/? 4 - 92/3 3 - 88/3 2 - 92/3 - 44 and so is negative when e (-1, 1). Hence, for e (-1, 1) 



13 which can be downloaded from http://www.maths.warwick.ac.uk/~strien/Publications 
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the cubic (5.23) has a unique real root v, and this root is in (0,1). Moreover, this solution v 
depends continuously on (3. When (3 = 0, this solution satisfies the inequalities that guarantee the 
behaviour is as shown in the figure: indeed when (3 = then we obtain m = 0.594.., n 2 = 0.129.., 
n 3 = 0.277.., mi = m 2 = 0.405.., m 3 = 0.188... 

Let us show that equalities can only hold in (5.14) when (3 ^ (—1, 1) or (3 = a. If mi = m 3 
then the expressions for mi and m 3 give vp = (2 + (3)/(3 + 3(3) and then 

, , , (/3 2 + /?-l)(l + /? + /3 2 ) 2 

= ^TW = ' 

this implies that either (3 ^ (—1, 1) or (3 is equal to the golden mean a. If n\ = n 3 then because 
of the first equation in (5.17) the first two components v A (ti + t 2 ) in (5.15) are equal, and so 
(5 = 0; using these two equation again we get n\ — n 2 = and with 5 = this implies mi = m 3 
and therefore as we saw (3 £ (— 1, 1) or (3 = a. If n\ = n 2 then X = 5 + 1 + (3 and 



■j A (ti+h) = ^ \ //j.ci + ») + <) 



m(i + /3) 

l(1+/?) + 
n 3 (l+(3)+(3S 




which gives that <5n 3 + 6 = (36 and so either 5 = 0orn 3 = /3 — 1 and so f = ni = ri2 = 1- The 
latter is impossible because //3(1) > while the latter implies that m 2 = m 3 ( we assumed that 
fii = n 2 ); as we showed above m 2 = m 3 implies (3 ^ (—1, 1) or /3 = <r. 

It follows that there exists a symmetric clockwise periodic orbit for all (3 € (— 1, a), depending 
continuously on (3. As (3 f cr this periodic orbit shrinks to the equilibrium solution ni = n 2 = 
ra 3 = (1 + /?)/3 and mi = m 2 = m 3 = (1 — (3)/3. To see this, notice that fp((l + (3)/3) = is a 
fifth order equation which can be factorized as 

i(l + /3)(/? 2 +/3-l)(/3 2 -/3+l) = 

which is zero if (3 = a or (3 ^ (—1, 1). If (3 e (cr, 1) the inequalities in (5.14) no longer hold. This 
can be seen by considering the solutions for (3 = 1 in (5.18-5.22). 

5.2 Existence of the anticlockwise periodic orbit when (3 G (cr, 1). 

We first show that for a < (3 < 1 there exists a unique symmetric anticlockwise periodic orbit as 
described in Theorem 3.3in the main text. 

For such an orbit to exist, with the appropriate symmetry, the vectors v A and v B must behave 
as shown in Figure 10 below. 

We compute only the part of the orbit where player B prefers strategy 2, starting where player 
B is indifferent between strategies 2 and 3, and ending where player B is indifferent between 
strategies 1 and 2. During this interval, player A prefers first strategy 1 and then strategy 3. 
We then use the rotational symmetry to determine whether this corresponds to a periodic orbit. 
The complete orbit consists of three such sections, with the coordinates permuted cyclically on 
each section. 

As before, to avoid too many sub and superscripts, let n = (ni,n2,n 3 ) T be the initial value 
^(O), and m = (mi,m,2,m 3 ) be the initial value v B (0). So the starting position given in the 
figure satisfies 

rii > n 2 , n 3 and m 2 = m 3 > mi , (5.25) 



28 



o h 



h + t 2 



Figure 10: Components of v A and v B plotted against time for one third of the symmetric (anticlockwise) 
periodic orbit which exists for j3 £ (c, !)■ Note that the identity of the maximal coordinate of v A 
determines the motion of v B , and vice versa. Since v B is maximal throughout, v A (t) moves towards 
A2 = (0, 1, (3) T in both time intervals. In the interval [0, ii] coordinate v A is maximal and v B (t) moves 
towards B\ — (— j3, 1,0); in the interval [ti,ti + £2], v A is maximal and v B (t) moves towards B3 = 
(1,0,-/3). 



and the orbit in v A ,v B space initially heads to respectively (0, 1,/3) T = A 2 and (— /?, 1,0) = B\ 
as we require. The first time that the orbit meets Z (the indifference set) again in t > is when 
the first and third coordinates of 




(t) = (l-t)n + t\ 1 (5.26) 



become equal and player A becomes indifferent between strategics 1 and 3. Solving v A (t) = vf(t) 
in (5.26) for t = t\ > 0, we obtain 

n\ - n 3 + )3 

Of course, depending on the values of the rii it is possible that we first get that n\ (t) = n 2 (t) for 
some t € (0,ti). Since we are not interested in this case, we insist that this docs not occur and 
so our initial condition satisfies 

(m -n 2 )/3 > (m -n 3 ). (5.28) 

Note that the motion of v B ensures that the largest coordinate of v B must be v B for the whole 
of the period (0,ti), so there is no danger that player B becomes indifferent before player A. 
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The v A and v B values at t = t\ are, using (5.13), 

1 

(m - n 3 + p) 



r A (h)-- 1 ; m ( /3n 2 +ni-n 3 ) (5.20) 



and 

uB ( < i) = 7 — 5 r(pm 1 -p(n 1 -n 3 ), pm 2 + (m - n 3 ) , /3m 3 ) . (5.30) 

(ni - n 3 + /?) 

After this moment, player A changes direction and so v B = p A B heads for (1,0,-/3) and t 2 > 
becomes the smallest number so that the first and second coordinates of v B [t\ + t 2 ) = (1 — 
t 2 )v B (ti) + t 2 (l, 0, — (3) become equal. This occurs when 

t> 2 B (*i)-if(ti) 0(m 2 - mi ) + (1 + P)(m - n 3 ) 

2 - vf{H) + 1 /3(m 2 -m 1 ) + (2 + /3)(n 1 -n 3 )+/3- {m ' 

Hence, writing 

5 = P(m 2 - mi) + (1 + P)(ni - n 3 ) 

we have 

t 2 



5 + P + ni - n 3 
Using (5.13) again, this implies 



1 



mP 

v A (h+t 2 ) = — a — | m - n 3 + n 2 p + S \, (5.32) 

ni/3 + /3<5 



<5 + /? + ni - n 3 

and 



v B (h + t 2 ) = — - 1 ((/3m 2 + ni - n 3 ), (/3m 2 + m - n 3 ), /3(m 2 - 5)) . (5.33) 

o + p + n\ — n 3 

To get an anticlockwise symmetric periodic orbit we require that 

n 2 \ / m 2 

« A (ii+i 2 )=( « 3 and w s (ii+t 2 )= m 2 |. (5.34) 

ni / \ mi 

Combining (5.32), (5.33) and (5.34), we need to solve the following equations: 

ni ■ P = n 2 ■ (S + n x - n 3 + p) 

n\ — n 3 + n 2 ■ + S = n 3 • (5 + n\ — n 3 + P) 

m-P + P-S =ni-(6 + n 1 -n 3 + P) (5.35) 

m 2 ■ P + m - n 3 = m 2 ■ (5 + m - n 3 + p) 

m 2 ■ P - P ■ 5 = mi ■ (6 + m - n 3 + p) 

and, collecting together (5.25) and (5.28), we require solutions to (5.35) that satisfy the strict 
inequalities 

m 2 = m 3 > mi, n\ > max(n 2 , n 3 ) and (m — n 2 )p > n\ — n 3 . (5.36) 
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It is straightforward to check using the computer algebra package MAPLE, 14 or more labo- 
riously by hand, 15 that solutions to the equalities (5.35) satisfy: 

m 2 = m 3 = \i, 

mi = - 2 n, 

ni = - nj3 

2 2 + 1 - 3 n 3 - 5 y0 2 - 2 n - 2 n + 3 /i 2 2 + 3 M 2 + 3 3 



n 2 



n 3 



0+1 + 2 

2 -0-4fi0 3 - 6 a* /3 2 - 3^/3-2/1 + 3 /i 2 /? 2 + 3 /3 + 3 /x 2 /3 3 



/3 + 1 + /? 2 
where a* satisfies /^(/i) = and where 

/ (Z) = (3/? 2 + 3/5 + 3 /3 3 ) Z 3 + (-5/3 3 - 7 p 2 -4/3- 2) Z 2 + 

+ (1 + /3 + 5/3 2 + 2/? 3 ) Z - P 2 . (5.37) 

Note that for P e (0,1), .^(0) = -0 2 <0and/ /3 (l) = -2 < while fp (1/3) = (2/9)/? 3 + l/9 > 0. 
This implies that the cubic polynomial (5.37) always has three roots fi, two in (0, 1) and one 
in (l,oo). When = 1, the only of these three roots which leads to a solution satisfying the 
inequalities (5.36) has fi = 0.155.., so m = 0.844.., n 2 = 0.449.., n 3 = 0.706.., mi = -0.311.., 
and m 2 = m 3 = 0.155... This solution (which depends on /3) continues to exist as (j decreases, 
and gives a periodic orbit of the desired type until one of the strict inequalities (5.36) fails to 
hold. Let us check what happens if one of these inequalities fails, still assuming p e (0, 1). If 
mi = m 2 then by the expressions above, a* = (1 — P)/3. If n\ = n 2 then by the first and third 
equations in (5.35), 6 = and either m = n 2 = n 3 or n\ = and so /i = 1 (which is impossible 
because 1 is not a root of fp). If n\ = n 3 then the first three equations in (5.35) imply m = P 
or S = 0; if m — then using the formula for m we get = — \i0 and so either = or /i = 
(both of which are impossible) whereas if 6 — (and n\ = n 3 ) then mi = m 2 and so, from the 
previous argument, \i = (1 — /3)/3. If (ni — n 3 )/3 = ni — n 2 then manipulating the first three 
equations in (5.35) we get 1/(1 — 0) = (which is impossible since G (0, 1)) or 6 = which 
in turn implies ri\ = n 2 = n 3 or ri\ — n 2 + 1 = which (after some manipulations) gives ri\ = 
and so \i = which again is impossible. In other words, the only way the inequalities can break 
down is when \i becomes equal to /i = (1 — 0)/3. But if we substitute Z = (1 — 0)/3 into (5.37) 
we obtain a sixth order equation which can be factorized as 

^(/3 c 2 + /3 c -l)(/3 c + l + /3 2 ) 2 = 0. 

Hence the critical value C of is equal to the golden mean a (since the other solutions of 
the sixth order polynomial in are negative or complex). It is straightforward to check that 



14 which can be downloaded from http://www.maths.warwick.ac.uk/~strien/Publications 

15 If manipulating these equations by hand, it is useful to observe that the solution must satisfy 

ni + 712 + n3 = 1 + P and mi + ni2 + mz = 1-/3 

which come from the definitions of v A and v B and the fact that p A and p B are probability vectors. 
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for all (3 G (0, 1) the solution generated by the other root of (5.37) fails to satisfy the strict 
inequalities (5.36). Thus we have shown that there is unique symmetric anticlockwise periodic 
orbit as described in the proposition for (3 G (cr, 1] and none otherwise. Furthermore, the orbit 
shrinks down to the interior equilibrium as (3 J. a. 



5.3 Stability of the anticlockwise and clockwise periodic orbits 

As stated in the main text: 

Proposition 5.1. Shapley's periodic orbit is attracting for all (3 G [0, cr). The anticlockwise orbit 
which exists for [3 G (cr, 1) is attracting when (3 G (r, 1) and of saddle-type when (3 G (cr, r). Here 
t is a root of a specific expression whose value is a little larger than 0.915. 

Proof. Let us just show the computation for the anticlockwise periodic orbit, because in that case 
an interesting bifurcation occurs. Since J2 n i = 1 + P an d 2 m « = 1 — we shall perturb the 
initial conditions considered in Section 5.2 above to n\ = n\ + ei, n' 2 = n 2 + e 2 , n' 3 = n 3 — ei — e 2 
while vn\ — m\ — 2e 3 , m' 2 = m' 3 = m 2 + £3. Repeating the calculation of Section 5.2 with these 
perturbed values we find that after slightly altered times t[ and t' 2 we arrive at a final position 
v A (t' 1 + t' 2 ) with coordinates v^(ti + t 2 ) + 7i, v 2 (t\ + t 2 ) + 72, v$(ti + t 2 ) — 71 — 72 and a final 
position for v B (t[ + t' 2 ) with coordinates (ti + t 2 ) + 73, vf (ti + t 2 ) + 73, v 3 {t\ + t 2 ) — 273 
where 

/3(3 + 2(3) - m2(2 + (3) [3(1 - m + [3) - 2n x 3/3 2 - 3mf3 \ / £l 
(3^2n 2 (2 + (3) -n 2 {2 + (3) -3n 2 (3 e 2 

2 _ 2(^n,)(2 + ffl 1 W- ni )(2+p) _ w ) y ^ 

plus higher order terms. 16 One can compute the eigenvalues of this matrix using MAPLE; one 
of of them is n 2 /ni, while the others are given as rather complicated (but exact) formulae in m, 
n 2 and (3: 

n 2 (-2TH/3 - m - 2n 2 - n 2 f3 + 2(3 2 + ±y/A) 

'-, (5.38) 




2ni/3 



where 



A := 10nin 2 /3 + 4nin 2 - 4n 2 /3 2 - 4n 2 /3 3 - 8m (3 3 + 4/3 4 + 4n 2 2 + An 2 2 (3+ 

4m 2 /3 2 + An 1 2 (3 + n 2 2 2 + 4 n 2 /3 2 ni - 8 m /3 2 + m 2 - 4 n 2 /3 - 8 m /3 + 4/3 2 + 4 /3 3 

Using the explicit formulae for ni computed in Section 5.2, and by inserting these in the ex- 
pressions for the eigenvalues, we find that the eigenvalues are expressions in (3. When [3 = 1, 
these are 0.532.., —0.815.., and —0.184.. (where the first one is n 2 /n\). One can check that as (3 
varies the eigenvalues (5.38) always remain negative. By numerically evaluating the expressions 
for these (which can be done as accurately as required) we find that one of them crosses — 1 
when (3 is equal to r where t is roughly 0.915 while the other remains, for all (3 G (cr, 1), in 
(—1,0). Consideration of the sizes and signs of the eigenvalues described shows that for (3 > r 
the periodic orbit is attracting, and for [3 < t it becomes of saddle-type. 



16 This matrix can be deduced from equations (5.35), where it is convenient to observe that we can use the first 
of those equations to write (<5 + n\ — 713 + 0) = /3ni/n,2- 
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Note that we do not have a generic periodic doubling bifurcation 17 when one of the eigenval- 
ues becomes equal to —1. Indeed, at this parameter the linear part of the above transformation 
has an eigenvector V associated to the eigenvalue — 1. However, the transition map which sends 
the hyperplane mi = m% to the hyperplane m\ — m 2 is a composition of central projections 
(see the proof of Theorem 3.2. In particular, it sends the line through V to the line through 
V (remember V is an eigenvector) and it is a linear fractional transformation on this line with 
derivative —1 at the fixed point. But taking a map of the form f(x) = —x + ax 2 + 72; + ... we 
always have f 2 (x) = x + Ox 2 + (?)x 3 . So in our case we get that the second iterate of the linear 
fractional transformation is equal to the identity map up to second order at the fixed point. But 
this implies that it is in fact equal to the identity map. So instead of a generic periodic doubling 
bifurcation, in this case we get non-generic dynamics at the bifurcation parameter (3 = r; at this 
parameter value there is a one-parameter family (continuum) of periodic orbits. 

The computation showing that Shaplcy's periodic orbit is attracting for all (3 e (0, a) goes 
similarly. □ 



5.4 Dynamics and a periodic orbit on the set J 

We will now consider, again using v A , v B space, the flow on the codimension two indifference set 
J defined in the main text. Recall that the flow which remains on J is the unique continuous 
extension to J of the flow off J, everywhere except at the interior equilibrium. 

Proposition 5.2. The system has a Hopf-like bifurcation when (3 = a on the topological two- 
dimensional manifold J. The orbits on J spiral towards the interior equilibrium E = (E A ,E B ) 
when (3 G (0, a] while for (3 G (a, 1) all orbits in J (except the one remaining at E) tend to a 
periodic orbit T on J. 



Proof. Again we compute a segment of orbit which makes one third of a complete rotation around 
the interior equilibrium, and use the rotational symmetry to compute the whole orbit. We take 
an initial condition v A = n and v B = m where ni = n.3 > ni and mi = = = (1 — /3/3). 

Player A (whose preferences depend on v A ) is at first indifferent between strategies 2 and 
3, and so adjusts p A towards some convex combination of P 2 and P 3 , implying that v B moves 
towards the same convex combination of B 2 and B 3 . The convex combination 

is chosen so that the orbit remains in J, i.e. so that vf (t) = v B (t) in t > 0. (We stress again that 
this is not in any sense an arbitry choice; it is the unique choice giving a continuous extension 
of the flow defined off the codimension two indifference surface.) Consequently player B will, for 
t > 0, be indifferent between strategies 1 and 3, will adjust p B towards some convex combination 
of P B and P B , and so v A will move towards the same convex combination of A\ and A 3 . In this 
case, the combination 

1 ^i + 77^T^3 = 7T^T( 1 + / 32 ^^) T ( 5 - 4 °) 



(1+13) (1 + /3) (1 + 3) 



17 as would be expected in a normal differential equation 
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n 2 = n 3 



mi = m,2 = m,3 = Ul - (3) 




Figure 11: Part of the periodic orbit on J with (v A ,v B ) plotted against time. 



is chosen so that v A {t) = v A (t) in t > 0. 

This motion continues until a time t\ € (0, 1) at which v A (t\) = v A (t\) = v A {t\) = |(1 + /3) 
so that t\ satisfies, using (5.13) and (5.40) 



v A (t 1 ) = (l-t 1 )v A (0)+t 



1 



1 + /3 2 

(3 



1 + 



Hence 



h = 



1 + P 
(n 2 -m)(l + /3) 



(n 2 -m)(l+/3) + (l + /3 2 -/3) 
We have also, from (5.13) and (5.39), 



1 



v B (t 1 ) = v B (0)(l-t 1 ) + t 1 ^{l, -/3-/3 2 , 1) 



Now since vf (0) = (1 — /3)/3 for i = 1, 2, 3, we have 



<(t 1 )-<,f(i 1 ) = _L^[l + /? + /3 2 ]. 



(5.41) 



Player A now becomes indifferent between strategies 1 and 3, while player B is still indifferent 
between strategies 1 and 3 (inspection shows this is the only way to remain on J) . In the interval 
t G (ti,ti +t 2 ) we have that v A (t) heads for 



(1-/3) 



(2-/3) (2-/3) ° (2-/3) 



M = -^r(l,/3(l-/3), 1) T 



(5.42) 
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and v (t) heads for 



! ' + % + tt^Ss = (-/3 2 , 1 + /3, -/3 2 ) . (5.43) 



(1 + 2/3) (1 + 2/3) J (1 + 2/3) 
Using (5.13) and (5.43) we seek to compute t 2 such that w B (ti + t 2 ) = (1, 1, 1). This gives 



*2 

which implies 



-(-/3 2 , 1+/3, -/3 2 )-t; B (t 1 ) 



(1 + 2/3) 

v?(h)-vi(t 1 ) ti(l + 2/3) 



3 

^(1 -/?)(!, l,l)-v B (ti) 



*2 = 



I±|^+<( tl )-uB( tl ) ti(l + 2/3) + (2 + /3)- 
Now (5.13), (5.42) and the fact that all components of v A {t\) are equal imply that 

v A (h + t 2 ) - ^(tx + t 2 ) = ^ [1 - /3 + /3 2 ] . 

Substituting in for t 2 and then for t\ we obtain 

« x (ti + ta) - (ti + 1 2 ) - ( , i(1 + 2/3) + (2 + /3)) (2,^ = 

= (n 2 - ni) (l+/3)(l + 2/3)(l-/3 + /3 2 ) 

(n 2 -m)3(l + /3) 2 + (2 + /3)(l-/3 + /3 2 ) (2-/3) 

This formula gives us the value of \v A (t) — v^WI at the end of the two legs, as a function F() of 
the initial value \n 2 — ni|, where 

F(x) . = X(l + /3)(l + 2/3)(l-/3 + /3 2 ) 



(2 - /3) (3(1 + (3) 2 X + (2 + /3)(1 + /3 2 - /3)) ' 

aX 
bX+c 



Now F is a linear fractional transformation F(X) = °^ and 



a (l + /3)(l + 2/3)(l-/3 + /3 2 ) 2/3 2 + 3/3 +1 
lJ c (2-/3)(2 + /3)(l-/3 + /3 2 ) -/? 2 +4 

which is less than 1 when (3 < a and greater than 1 when /3 > a. Since F(X) — > 1 as X — > oo, 
this implies that a; i— » -F(x) has zero as an attracting fixed point when /3 e (0, a], and there exists 
an attracting fixed point x > of F when (3 > a. This solution x corresponds to the value of 
n>2 — Tti for which there is an associated periodic orbit T on J. This orbit attracts all orbits on 
J except the one starting at the interior equilibrium (E A , E B ). 

To prove the last past of the proposition, note that F(X) = X is equivalent to 

x _ (1+/3 2 - m 2 

(2-/3)(l + /?) 2 

Now for the initial condition n on T, we have n 2 — n\ = X = F(X), n 2 = n 3 , and m + n 2 + n 3 = 
(1+/3). Hence m = ~ 2X ^" 1+;3 . Note that on the line Z A 3 in (see Figure 9 in the main text) one 
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has (inp B coordinates) Qf 3 = 1 +/3 2 , P), E B = ±±£(1,1,1) and Rf 3 = ^(1, (3 - /3 2 , 1) 

(and symmetrically for other points). During the third of the orbit studied we have that we 
travel first along [R B 3 ,E B ] until meeting E B and then along [E B ,Rf 3 ]. The proportion 

\^23 ~ fxi\ 



\Q"23 

is therefore, (by doing the computations in the p B -simplex) 



1+/3 2 „ , 1+/3 2 -2X + 1+/3 2/3+1 ' 

1+/3 111 1+/3 3 M 

So when (3 is equal to the golden mean then this is equal to and it increases monotonically 
when {3 € [a, 1) (its maximum is 1/3). Similarly, 

\R^-E^\ i±^-|Ef Y-^f (i + Z 3 ) 2 

(which again increases in /3). □ 

Note that when (3 ^ a, the time taken for a trajectory on J to spiral into or out of the interior 
equilibrium E is finite. This is because for small X we have that F(X) = c\((3)X + 0(X 2 ) and 
that the time taken for this step is ti + t 2 = c 2 (f3)X + 0(X 2 ) for some constants c\{(3), C2{(3) 
depending on (3. When c\(f3) ^ 1, which is true precisely when (3 ^ a, this gives geometric 
(rather than exponential) convergence towards or divergence away from E. This phenomenon is 
intimately connected with the fact that the flow does not extend in a continuous way to E; there 
is genuine non-uniqueness of the flow at E (since trajectories may remain at E for all positive 
and negative time, or may leave it at some moment in positive time ((3 > a) or in negative time 
{(3 < a). When (3 = a the time taken for trajectories to tend to E is infinite, as can easily be 
checked. 



36 



0.64 




4.5 



time 



